On Low-Space Differentially Private Low-rank Factorization in the Spectral Norm

Low-rank factorization is used in many areas of computer science where one performs spectral analysis on large sensitive data stored in the form of matrices. In this paper, we study differentially private low-rank factorization of a matrix with respect to the spectral norm in the turnstile update model. In this problem, given an input matrix A ∈ Rm×n updated in the turnstile manner and a target rank k, the goal is to find two rank-k orthogonal matrices Uk ∈ Rm×k and Vk ∈ Rn×k, and one positive semidefinite diagonal matrix Σk ∈ Rk×k such that A ≈ UkΣkVT k with respect to the spectral norm. Our main contributions are two computationally efficient and sub-linear space algorithms for computing a differentially private low-rank factorization. We consider two levels of privacy. In the first level of privacy, we consider two matrices neighboring if their difference has a Frobenius norm at most 1. In the second level of privacy, we consider two matrices as neighboring if their difference can be represented as an outer product of two unit vectors. Both these privacy levels are stronger than those studied in the earlier papers such as Dwork et al. (STOC 2014), Hardt and Roth (STOC 2013), and Hardt and Price (NIPS 2014). As a corollary to our results, we get non-private algorithms that compute low-rank factorization in the turnstile update model with respect to the spectral norm. We note that, prior to this work, no algorithm that outputs low-rank factorization with respect to the spectral norm in the turnstile update model was known; i.e., our algorithm gives the first non-private low-rank factorization with respect to the spectral norm in the turnstile update mode. Our algorithms generate private linear sketches of the input matrix. Therefore, using the binary tree mechanism of Chan et al. (TISSEC: 14(3)) and Dwork et al. (STOC 2010), we get algorithms for continual release of low-rank factorization under both these privacy levels. This gives the first instance of differentially private algorithms with continual release that guarantees a stronger level of privacy than event-level privacy.